os: start processes with posix_spawn - #5634
Open
yohimik wants to merge 2 commits into
Open
Conversation
This was referenced Aug 30, 2026
The third parameter of fcntl is variadic, and on darwin/arm64 a variadic argument goes on the stack and not in a register. A call to libc fcntl through a plain three-argument function pointer thus makes the callee read that argument from an unrelated stack slot. open() already has a C wrapper for the same reason. The symptom is quiet. fcntl(fd, F_SETFD, FD_CLOEXEC) sets the flag or does not, which depends on the stack contents, so the result is the same for one binary and different between binaries. syscall.CloseOnExec is the main caller, so when it fails, every descriptor of the program goes into every process that it starts. A child that holds a copy of the write end of a pipe keeps that pipe from a report of EOF, which is how os/exec collects the output of a command. Measured on macOS 26.6 arm64 before this change, fcntl(fd, F_DUPFD, 100) returns EINVAL, and three F_SETFL calls with 0x4, 0x0 and 0x8 all leave F_GETFL with 0x48. The wrapper takes the argument as a uintptr_t so that the pointer commands reached through syscall.fcntlPtr use it too. Both spellings go through libc_fcntl_trampoline, and on a little-endian target the int commands read the low half of the same stack slot. The new tests in src/os cover both shapes. TestFcntlSetNonblock fails on darwin before this change and passes after it.
The process layer was a stub. StartProcess refused every ProcAttr that carried Dir, Sys or Files, and os/exec always passes three Files, so no command could run. Wait, Kill and Signal returned ErrNotImplemented, and ProcessState was an empty struct whose methods all reported a failure. The code that did exist was a fork() and an execve() with no branch on the result of the fork, so the parent fell into the exec as well. Use posix_spawn(3) on hosted Linux and macOS, which are the two targets where the standard library os/exec and syscall packages compile against this override. Those targets run the threads scheduler and collect with Boehm, so a fork from Go gives the child one thread that holds the locks of the other threads, malloc among them, and the stop-the-world signal of the collector can arrive between the fork and the exec. posix_spawn does the clone and the exec inside libc, where no Go code runs, and it reports a failed exec as its return value, so the usual status pipe is not necessary. The descriptors of the child come from a file-actions list. There is a dup2 for each entry of ProcAttr.Files, a close for a missing one, and an addchdir_np for Dir. A nil Env means the environment of the parent, as Go documents. The attribute block installs an empty signal mask, because a blocked mask survives an exec and the spawning thread can carry the signal of the collector blocked. Setpgid and Pgid are honoured through posix_spawnattr_setpgroup, which is the one SysProcAttr request that posix_spawn can express. Every other field is refused by name, and the error unwraps to ErrNotImplementedSys. Wait reaps with wait4 and retries on EINTR, which a thread in wait4 gets as a matter of course, because the collector interrupts it. ProcessState now carries the pid and the real syscall.WaitStatus, so exec.ExitError reports "exit status N", and ExitCode, Exited, Success and Sys work. A killed child is reported as signalled. Signal refuses a pid that Wait reaped and maps ESRCH to ErrProcessDone, which is what exec.CommandContext expects when its context fires as the command finishes. macOS has no pipe2, so os.Pipe there marks both descriptors close-on-exec afterwards, under ForkLock. Without the flag every pipe goes into every child, and a child that holds a copy of a write end keeps that pipe from a report of EOF. Linux asks for O_CLOEXEC in pipe2 and gets it atomically. The minimal macOS SDK in lib/macos-minimal-sdk does not declare <spawn.h>, so the generated libSystem stub has none of the posix_spawn symbols and a darwin program that starts a process does not link. The builder now assembles the missing names into a second stub object. posix_spawn_file_actions_addchdir_np came with macOS 10.15, so a binary from this toolchain needs at least that release. Targets without a process model keep the previous stubs. Only the build tag on exec_other.go changes, to let macOS through to the new implementation.
yohimik
force-pushed
the
upstream-pr/os-exec-posix-spawn
branch
from
September 2, 2026 08:50
628c5e6 to
42486b3
Compare
Author
|
Rebased on dev after the 0.42.0 release. The change applies on top of v0.42.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
os: start processes with posix_spawn
Depends on
upstream-pr/darwin-fcntlon darwin. See "Dependencies".What this does
The process layer was a stub.
StartProcessrefused everyProcAttrthatcarried
Dir,SysorFiles, andos/execalways passes threeFiles, sono command could run.
Wait,KillandSignalreturnedErrNotImplemented,and
ProcessStatewas an empty struct whose methods all reported a failure. Thecode that did exist was a
fork()and anexecve()with no branch on theresult of the fork, so the parent fell into the exec as well.
This uses
posix_spawn(3)on hosted Linux and macOS, which are the two targetswhere the standard library
os/execandsyscallpackages compile against thisoverride.
Why
posix_spawnand not fork plus exec. Those targets run the threadsscheduler and collect with Boehm, so a fork from Go gives the child one thread
that holds the locks of the other threads,
mallocamong them, and thestop-the-world signal of the collector can arrive between the fork and the exec.
posix_spawndoes the clone and the exec inside libc, where no Go code runs,and it reports a failed exec as its return value, so the usual status pipe is
not necessary.
What the change covers.
dup2for each entry ofProcAttr.Files, aclosefor a missing one and for a standard descriptorthat
Filesdoes not name, and anaddchdir_npforDir.Envmeans the environment of the parent, as Go documents.survives an exec and the spawning thread can carry the signal of the collector
blocked.
SysProcAttr.SetpgidandPgidare honoured throughposix_spawnattr_setpgroup. Every other field is refused by name, and theerror unwraps to
ErrNotImplementedSys.Waitreaps withwait4and retries on EINTR.ProcessStatecarries the pidand the real
syscall.WaitStatus, soexec.ExitErrorreports "exit statusN", and
ExitCode,Exited,SuccessandSyswork.Signalrefuses a pid thatWaitreaped and maps ESRCH toErrProcessDone,which is what
exec.CommandContextexpects when its context fires as thecommand finishes.
os.Pipeon darwin marks both descriptors close-on-exec, underForkLock,because macOS has no
pipe2. Linux asks forO_CLOEXECinpipe2.posix_spawnfamily in the darwin libSystem stub.The minimal macOS SDK in
lib/macos-minimal-sdkreads a fixed list of headersthat does not have
<spawn.h>, so a darwin program that starts a process didnot link.
Targets without a process model keep the previous stubs. Only the build tag on
exec_other.gochanges, to let macOS through.Evidence
src/os/exec_linux_test.go, which asserted the not-implemented errors, isreplaced by
src/os/exec_spawn_test.go, which asserts the behaviour and alsocovers darwin. The
ospackage is already inTEST_PACKAGES_FAST, so these runin the linux and the macOS CI jobs with no makefile change.
Thirteen tests cover exit status, a non-zero exit, kill and the signalled
status,
ErrProcessDoneafter a reap,ErrNotExist,Dir, an emptySysProcAttr, every refusedSysProcAttrfield,Setpgidwith a new group,Setpgidjoining a group, the inherited group, the close of a standarddescriptor that
Filesdoes not name, descriptors that must not leak past theexec, and
Fileshanded to the child.All pass on macOS 26.6 arm64 with
tinygo test os.A downstream product ships binaries built with these changes in a production
release. dispat v1.4.0 is published and is not a prerelease. It carries
dispat-tiny-linux-amd64anddispat-tiny-linux-arm64, built by the forkrelease v0.42.0-net.4 from sha256-pinned tarballs and smoke-executed under
binfmt before upload, beside six binaries from the gc toolchain.
https://github.com/yohimik/dispat/releases/tag/services%2Fdispat%2Fv1.4.0
The acceptance record of that repository is committed at
packages/docs/docs/internals/tinygo.md. It reports the net.2 to net.4acceptance history, an integration suite of 694 rows that passes with 0 failures
and 1 documented skip on darwin, and a size table of 0.58x to 0.63x against the
gc equivalents with TinyGo
-opt=z -no-debugagainstgo build -trimpath -ldflags "-s -w". Those figures come from that document. They are not ameasurement of this branch.
The suite exercises process spawning with pipes, file I/O, environment
variables, goroutine concurrency under the threads scheduler, and time and
context handling.
Dependencies
upstream-pr/darwin-fcntl.os.Pipeon darwin marks thedescriptors close-on-exec with
fcntl, whose variadic argument is not passedcorrectly on darwin/arm64 without that fix, so
TestForkExecDescriptorsDoNotLeakis not reliable without it. Merge the fcntl PR first, or take the two together.
pipe2setsO_CLOEXECatomically.syscall.ForkLockis anRWMutex, and a program that spawns andmakes pipes at the same time can stop on the current
RWMutex. Seeupstream-pr/sync-rwmutex. It is not needed for the tests here, which aresequential, but it is needed for real concurrent use.
Known gaps
SysProcAttrfields other thanSetpgidandPgidare refused, each byname. Every one of them needs Go code to run in the child between the clone
and the exec, which is what
posix_spawndoes not offer.posix_spawn_file_actions_addchdir_npcame with macOS 10.15, so a binary fromthis toolchain needs at least that release. The deployment target is lower.
Processgains a 4-bytedonefield that targets without a process model donot use.
ProcAttr.Files. A sourcedescriptor that is numerically below its own destination index is thus
overwritten before a later entry can read it.
syscall.forkAndExecInChildin the standard library moves such a source outof the way first.
os/execnever builds a layout of that shape, because thedescriptors that it passes come from pipes and are above 2, so this only
affects a direct
StartProcesscall with an unusualFilesslice. It can bea follow-up, or it can be added here if the maintainers prefer.
Related
os/StartProcess #4377 added
StartProcesswith fork and exec.Files,Waitand darwin werestill missing after it, which is what this completes.
Related pull requests
This change is part of one body of work. Together the changes make programs that use the network and child processes work on hosted linux and macOS. A full CLI was tested end to end with all of them and ships binaries built this way, see dispat v1.4.0 in the evidence section.
In this repository
In tinygo-org/net
A merge order that works. The three bug fixes are independent. #5633 goes before #5635. HTTPS on linux needs only #5633 and #5635. Full darwin support also needs #5636, the net changes and a new src/net submodule pin.